Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=36186136013 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=36186136013 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and found no bugs. Because it touches configs/nvidia-master.yaml (a CODEOWNERS-protected file) and adds a new model-branch to a benchmark script with several performance-tuning constants, a human look would still be worthwhile.
What was reviewed:
- The new
$MODEL-conditional branches in dsv4_fp4_b300_sglang_mtp.sh (SPEC_ARGS/ROUTER_ARGS refactor, HiCache ratio, prefill-decode-interval, chunked-prefill flags, MEM_FRACTION_STATIC). - Router failure isolation:
--disable-health-checkalready applies to both model variants (line ~310), so the new EAGLE-only--disable-circuit-breakerdoesn't remove an existing guard. - The unconditional
MEM_FRACTION_STATIC=0.84for DeepSeek-V4-Pro sits inside the TP>=8 branch's sibling scope, but the new nvidia-master.yaml recipe only exercises tp:8, so the documented TP<8 (DEP4) 0.902-minimum concern isn't reached by this config. - perf-changelog.yaml addition is appended at the tail, consistent with the append-only convention.
Extended reasoning...
The diff extends a B300/FP4/SGLang agentic benchmark script with a second model branch (EAGLE spec-decoding vs DSPARK), adds one new recipe to the CODEOWNERS-protected configs/nvidia-master.yaml, and appends one perf-changelog entry at the tail (no rewritten history). No security-sensitive surface (auth, crypto, permissions) is touched; the risk is purely in benchmark tuning correctness (spec-decoding params, mem-fraction overrides, router flags). Deciding factor for defer over approve: configs/nvidia-master.yaml is owned by specific CODEOWNERS, and the change carries several magic tuning constants whose correctness can't be fully validated from the diff alone.
This review covers commit e83cbd3, which is no longer the latest commit on this pull request; later commits are not covered by it.
更新 B300 DSV4 性能。
补充 B300 DSV4 TP8 低延迟并发点 1、2、4、8、16、32。
DSV4 低延迟配置使用明确的 decode CUDA graph 参数，修复新镜像的 CLI 歧义错误。
添加 B300 DSV4 TP4 低延迟测试点，并保留现有 TP8 和 DP8 配置。
033cc79 to
2e46f34
Compare
|
InferenceX has switched away from unmaintainable bash scripts to YAML files that don't repeat the same stuff over and over again. Please merge the latest |
This branch has not been deployed
2 participants
B300 curve update